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Abstract 

A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equa- 
tions with deterministic coefficients. By a variational method, the optimality system is derived, which 
turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling 
technique, two Riccati differential equations are obtained, which are uniquely solvable under certain 
conditions. Then a feedback representation is obtained for the optimal control. 
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1 Introduction. 

Let (ri,J-", P,F) be a complete filtered probability space, on which a one-dimensional standard Brownian 
motion W{-) is defined with F = {Tt}t>o being its natural filtration augmented by all the P-nuU sets. 
Consider the following controlled linear stochastic differential equation (SDE, for short): 

dX{s) = |a(s)X(s) + B{s)u{s) + A{s)'E[X{s)] + B(.s)E[m(s)] jds 
' +{Ai{s)X{s) + Bi{s)u{s)+Ai{s)E[X{s)] + Bi{s)E:[u{s)]}dW{s), se[0,T], (^-l) 

X{0) = X, 

where A{-), B{-), A{-), B{-), Ai{-), Bi{-), Ai{-), Bi{-) are given deterministic matrix valued functions. In the 
above, X{-), valued in R", is the state process, and u(-), valued in R™, is the control process. 

We note that E[X(-)] and E[m(-)] appear in the state equation. Such an equation is referred to as a 
linear controlled mean- field (forward) SDE (MF-FSDE, for short). MF-FSDEs can be used to describe 
particle systems at mesoscopic level, which is of great importance in applications. Historically, the later- 
called McKean-Vlasov SDE, a kind of MF-FSDE, was suggested by Kac [18] in 1956 as a stochastic toy 
model for the Vlasov kinetic equation of plasma and the study of which was initiated by McKean [24] 
in 1966. Since then, many authors made contributions on McKean-Vlasov type SDEs and applications, 
see, for examples, Dawson [13], Dawson-Gartner [14], Gartner [15], Scheutzow [29], Graham [16], Chan 
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[10], Chiang [11], Ahmed-Ding [1], to mention a few. In recent years, related topics and problems have 
attracted more and more researchers' attentions, see, for examples, Verctennikov [32], Huang Malhame- 
Caines [17], Mahmudov-McKibben [23], Buckdahn-Djehiche-Li-Peng [8], Buckdahn-Li-Peng [9], Borkar- 
Kumar [6], Crisan-Xiong [12], Kotelenez-Kurtz [20], Kloeden-Lorenz [19], and so on. More interestingly, 
control problems of McKean-Vlasov equation or MF-FSDEs were investigated by Ahmed-Ding [2] , Ahmed 
[3], Buckdahn-Djehiche-Li [7], Park-Balasubramaniam Kang [28], Andersson-Djehiche [4], Meyer-Brandis- 
Oksendal-Zhou [25], and so on. This paper can be regarded as an addition to the study of optimal control 
for MF-FSDEs. 

For the state equation (1.1), we introduce the following: 

W[0,T] = L|(0,T;R'") = {u: [0,T] xfi^R" |E J \u{s)fds < 

Any «(•) e U[0, T] is called an admissible control. Under mild conditions, one can show that (see below) for 
any (a;, u{-)) £ R" x U[0, T], (1.1) admits a unique solution X{-) = X{- ; x, u{-)). We introduce the following 
cost functional: 



J{x;ui-)) = E 



; Q{s)X{s), X{s) ) + ( Q(,s)E[X(,s)], E[X(,s)] ) + ( R{s)u{s), u{s) ) 
{ R{s)E[u{s)], E[u{s)] )]ds + { GX{T),X{T) ) + ( GE[X(T)], E[X(r)] ) }, 



(1.2) 



with Q{-),R{-),Q{-),R{-) being suitable deterministic symmetric matrix-valued functions, and G,G being 
symmetric matrices. Our optimal control problem can be stated as follows: 

Problem (MF). For given x G R", find a u(-) e U[0, T] such that 

J(x]u(-))= inf J(x;u(-)). (1.3) 

Any u{-) G U[0,T] satisfying the above is called an optimal control and the corresponding state process 
X{-) = X{- •,x,u{-)) is called an optimal state process; the pair {X {■),€,{■)) is called an optimal pair. 

From the above-listed literature, one has some motivations for the inclusion of E[X(-)] and E[m(-)] in the 
state equation. We now briefly explain a motivation of including E[X(-)] and E[u(-)] in the cost functional. 
We recall that for a classical LQ problem with the state equation 



(1.4) 



dX{s)^ A{s)X{s)+B{s)u{s) ds+ Ai{s)X{s) + Bi{s)uis) dW{s), s e [0,T], 
X{0)=x, 
one has the following cost functional 

Mx; u{-)) = e{ [ ( Qo{s)X{s),X{s) ) + ( R<i{s)u{s), u{s) )]ds+{ GqX{T),X{T) ) }. (1.5) 

For such a corresponding optimal control problem, it is natural to hope that the optimal state process and/or 
control process are not too sensitive with respect to the possible variation of the random events. One way 

to achieve this is try to make the variations var [X(-)] and var [u{-)\ small. Therefore, one could include 
the var[X(-)] and var[u(-)] in the cost functional. Consequently, one might want to replace (1.6) by the 
following: 



Jo(a;;u(-)) =E 



( Qo{s)X{s),X{s) ) +g(s)var [X{s)] + { Rq{s)u{s),u{s) ) +p(s)var [u{s)] 
+ {GoXiT),XiT)) +gvar [X (T)] } , 



ds 



(1.6) 



for some (positive) weighting factors q{-), />(•), and g. Since 

var[X(.)]=E|X(.)|2-(E[X(,s)])', 
and similar things hold for var [X(r)] and var [w(s)], we see that 



T 



Jo(x;u(-))=e{ [{[Qo{s) + q{s)I]X{s),X{s)) -q{s)(E[X{s)]) + {[Ro{s) + p{s)I]u{s),u{s)) 
-p{s)(E[u{s)]y]ds + ([Go+gI]X{T),X{T))-g(E[X{T)]y}. 
Clearly, the above is a special case of (1.2) with 

Q{-) = Qo{-) + qi-)I, R{-)^R^{-) + p{-)I, G = Go + gI, 
Q{-) = -q{-)I, R{-) = -p{-)I, G = -gI. 

Note that in the above case, Q{-), R{-), and G arc not positive semi-definite. Because of this, we do not 
assume the positive semi-definiteness for Q{-), R{-), and G. 

The purpose of this paper is to study Problem (MF). We will begin with the well-posedness of the state 

equation and the solvability of Problem (MF) in Section 2. Then, in Section 3, we will establish necessary 
and sufficient conditions for optimal pairs. Naturally, a linear backward stochastic differential equation of 
mean-filed type (MF-BSDE, for short) will be derived. Consequently, the optimality system turns out to be a 
coupled mean- field type forward-backward stochastic differential equation (MF-FBSDE, for short). Inspired 
by the invariant imbedding [5], and the Four-Step Scheme [22], we derive two Riccati differential equations 
in Section 4, so that the optimal control is represented as a state feedback form. Well-posedness of these 
Riccati equations will be established. We also present a direct verification of optimality for the state feedback 
control by means of completing squares. In Section 5, we will look at a modified LQ problem which is one 
motivation of the current paper. 

2 Preliminaries. 

In this section, we make some preliminaries. First of all, for any Euclidean space H = R",R"^'",5" (with 
5" being the set of all {n x n) symmetric matrices), wc let LP(Q,t;H) be the set of all iJ-valued functions 
that are L^-integrable, p € [1, oo]. Next, we introduce the following spaces: 

Xt = i3r^(f2;R") = 1^ : f2 -)■ R" I ^ is Ji-measurable, E|^|2 < ooj, 
Ut = L|-^(f2;R") = |r? : f2 ^ R" I r? is Ji-measurable, E|r?p < ooj, 

L|(0,i;R") = |x : [0,t] x R" | X{-) is F-adapted, E^ \X{s)\^ds < ooj, 

X[0,t] = CF([0,i];R") = {x : [0,t] Xt \ X{-) is F-adapted, 

s H^- X{s) is continuous, sup E|X(s)|^ < oo L 

sG[0,t] 

X[0,t] = L^(n;C([0,t];R")) = |x : [0, t] x ^ R" | X{-) is F-adaptcd, 



X(-) has continuous paths, E sup |X(s)|^ <cxd[. 
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Note that in the definition of A'[0,f], the continuity of s h-> X{s) means that as a map from [0,t] to Xt, it 
is continuous. Whereas, in the definition of A'[0,t], X{-) has continuous paths means that for almost sure 
w G fi, s I— > X(s,uj) is continuous. It is known that 

X[0,t] C X[{),t] C Ll{0,t;IV'), X[0,t] / X[{)J.] / L|(0,t;]R"). 
We now introduce the following assumptions for the coefficients of the state equation. 
(HI) The following hold: 



A{-),A{.),A,{.),A,{.) e L-(0,T;]R"X"), 



(2.1) 



(HI)' The following hold: 

A{-),A{-) e L2(0,r;lR"^"), 
B{-),B{-) e L2(0,r;]R"^'"), 

(HI)" The following hold: 

yl(-),l(-) ei'(0,T;R"^"), 
S(-),B(-) eL2(o,T;R"^'"), 



Ai{-),Ai{-) £ L°°(O,T;]R"^"0, 
Bi{-),Bi{-) e L°°(0,r;]R"^™). 

eL2(o,T;R">^"), 
Bi(-),Bi(-) ei°°(0,T;R"^'"). 



(2.2) 



(2.3) 



Clearly, (HI) implies (HI)' which further implies (HI)". Namely, (HI)" is the weakest assumption among 
the above three. Whereas, (HI) is the most common assumption. For the weighting matrices in the cost 
functional, we introduce the following assumption. 

(H2) The following hold: 

g(-),g(-)ei°°(0,T;5"), i?(-)J(-)eL°°(0,T;5™), G,Gg5". 



(H2)' In addition to (H2), the following holds: 

Q{s), Q{s)+Q{s)>0, R{s), R{s) + R{s)>0, se[0,T], 
G, G + G > 0. 

(H2)" In addition to (H2), the following holds: For some 5 > 0, 

Q{s), Q{s) + Q{s) > 0, R{s), R{s) + R{s) >SI, s G [0,T], 
G, G + G > 0. 



(2.4) 



(2.5) 



(2.6) 



From (1.4), we see that Q{-), R{-), and G are not necessarily positive semi-definite. Therefore, in (H2), 
we do not mention positive-definiteness of the involved matrices and matrix-valued functions. 

Now, for any X{-) G L|(0,T;R") and any u{-) G U[0,T], we define 



[AX{-)]{t) = J (^A{s)X{s) + A{s)E[X{s)]ys 

+ (Ai(s)X(s)+Ai(s)E[X(s)])dW(s) 
[Bu{-)]{t)= ^ (^B{s)u{s) + B{s)E.[u{s)]^ds 

(Si(s)w(s) + Bi(s)E[u(s)])rfW(s), 



iG [0,T], 



tG [0,T]. 



(2.7) 



The following result is concerned with operators A and B. 



Lemma 2.1. The following estimates hold as long as the involved norms on the right hand sides are 
meaningful: For any t e [0, T], 



L2(0,t;E"X") ^ ll"*^lVllL2(0,t;E"X") 
l(')llLo°(0,t;R"X") + ll"^l(')llL°°(0,t;R"X") 



ll^(-)lli^(0,t;R'- 



\\A{-)f 



+ P(-)llii(0,;R" 



+ ll^l(-)lli.(0,t;R"x.) + Pl (■)llL2(o^t;]F."x,>) 



ll^(-)ll^[o.t]. 



and 



IIM-)ll|,,j<c 



!|-^(')lli2(0,(;R«Xr.) + l|5(-)lli2(0,t;R" 



+ ll^l(-)lli=c(0,*;Rn>c.) + ||5/^"2 



h(-)ll^[o,t]- 



(2.8) 



(2.9) 



(2.10) 



Hereafter, C > represents a generic constant which can be different from line to line. 
Proof. For any t e {0,T], and any X{-) e L|(0,t;R"), 



e( sup \[AX{-)]{s)\) <c|e( f \A{s)\\X{s)\d. 
^se[o,t] ^ >- V /n 



|l(s)||E[X(s)]|ds)' 



< 



< 



+E \A,{s)f\Xis)\^ds + \Ms)\''\nX{sWds} 

c{e[(^* \A{s)fds) (J^ \X{s)fds)] + (J^ \A{s)\'ds)(J^ \misWds) 

+ ( sup \Ai{s)\Y /*E|X(s)|2ds+ f sup |Ii(s)|2) /*E|X(s)|2rfs| 
^se[o,t] ^ Jo ^se[o,t] ^ Jo ' 

C{( f \A{s)\''ds) + ( f \A{s)\''ds) + sup |Ai(s)|2+ sup |li(s)|'| rE|X(s)|2ds. 
^^Jo ^ ^Jo ^ se[o,t] se[o,t] ^ Jo 



Thus, estimate (2.8) holds. Next, for any X(-) € X[0, t], making use of Burkholder-Davis-Gundy's inequality 
([34]), we have 

e(^sup^ \[AX{.)]{s)f) < c{e(^* \A{s)\\X{s)\dsy + [J^ |l(a)||E[X(5)]|ds)' 



< C 



< c 



\A{s)\ds 



\A{s)\ds) 



+E^* \Ar{s)\^\X{s)\^ds + ^* |ii(s)nE[X(s)]|2ds} 

|A(s)|E|X(s)pds) + ( f\Ais)\dsY( sup |E[X(s)]p) 

+ J* |Ai(s)|2E|X(s)|2ds + ^* |Ai(s)|2E|X(s)|2ds} 

\A{s)\dsy + I |Ai(s)|2ds+ / |li(s)|2dsl( sup E|X(s)p 
^ ^0 Jo -'^se[o,t] 



Hence, (2.9) follows. 

Similar to the proof of (2.8), we can prove (2.10). 
The above lemma leads to the following corollary. 



Corollary 2.2. If (HI)" holds, then A : X[0,T] X[0,T] and B : W[0,T] ^ X[Q,T] are bounded. 
Further, if (HI)' holds, then A : L|(0, T; R") X[0, T] is also bounded. In particular, all the above hold if 
(HI) holds. 

Next, we define 

ItX{-) = X{T), 

AtX{-) = ItAX{-) = [AX{-)]{T) 

= (^A{s)X{s)+A{s)E[X{s)])ds + (^A,{s)X{s)+MsMX{s)])dW{s), (2.11) 
Btu{-) = ItBu{-) = [Bu{-)]{T) 

= (^B{s)u{s) + B{s)E[u{s)]jds + (^Bi{s)u{s) + Bi{s)E[u{s)]jdW{s). 
It is easy to see that 

It : X[0,T] ^ Xt 

is bounded. According to Lemma 2.1, we have the following result. 

Corollary 2.3. If (El)" holds, then At : X[0,T] Xt and Bt : 1{[0,T] Xt are bounded. Further, 
if (HI)' holds, then At Lp{0, T; R") ->■ Xt is also bounded. In particular, all the above hold if (HI) lioWs. 

Recall that if ^ S Xt, then there exists a unique € ^^(O, T; R") such that 

C = EC+ rC{s)dW{s). 
Jo 

We denote 

C{s) = D,^, se[0,T], 

and call it the Malliavin derivative of ^ ([27]). Next, we have the following results which give representation 
of the adjoint operators of A, B, At, and Bt- 

Proposition 2.4. The following hold: 

{A*Y){s) = j^(A{sfY{t) + A{sfnY{t)] + A^{sfD,Y{t) + MsfW.[D,Y{t)\)dt, 

{B*Y){s) = j [B{sfY{t) + B{sfW,[Y{t)] + Bi{sfD,Y{t) + Br{sfW.[D,Y{t)])dt, (2.12) 

{A^O{s) = A{sf^ + A(s)^E^ + A^{sfD,i + li(s)^E[D,C], 
(B^O(s) = B{sfi + B{sf^^ + BiisfDs^ + B,{sfnD,^]. 

Proof. For any Y{-) G 1,2^(0, T;R"), 

{X,A*Y) ^ {AX,Y) = -E [ {[AX]{t),Y{t))dt 

Jo 

= E^ {J^(^A{s)X{s)+A{s)E[X{s)]'^d»j-j^ (^Ai{s)X{s)+Ai{s)E[X{s)]jdW{s),Y{t))dt 
= e[ [ {A{s)X{s)+A{s)E[X{s)],Y{t))dtds+E[ [ {Ai{s)X{s)+Ai{s)E[X{s)],D,Y{t)) dtds 

Jo J s Jo J s 

i-T 



= E^ {Xis),J^ [A{sfY{t)+A{sfW)[Y{t)]+Ai{sfDsY{t)+MsfE[D,Y{t)]yt)ds. 



Thus, the representation of A* follows. Similarly, we can obtain the representation of B*. 
Next, for any ^ e Xt, 



= E( (^A{s)X{s)+A{sMX{s)])ds,0+E{ (Ai(s)X(s) + li(s)E[X(s)])dM/(s),0 

= E / {X{s),A{sf^ + A{sf'R^ + Ar{sfD,^ + M{sf'E[D,^\)ds. 
Jo 

Therefore, the representation of A^ follows. Similarly, we can obtain the representation of By. □ 
For completeness, let us also prove the following result. 
Proposition 2.5. It holds 



where J^y} is the Dirac measure at T, and 



ad 

E*a; = a;'^E, Va; e R". 



(2.13) 



(2.14) 



Proof. First of all, since It : A'[0,T] Xt is bounded, we have : X^ = Xt X[0,T]*. For any 
^ G Xt, and any Y{-) G X[0, T], we have 

( /^e, Y{-) ) = ( ItY{-) ) = E ( ^, Y{T) ) = E r { Y{s),^ ) dTids). 

Jo 

Next, since E : Xt — )• R is bounded, we have E* : R — )• Xt- For any ^ € and x € R, 

(E*a;,C) = (a;,EO =E(a;,,e) =a;^E^. 
This completes the proof. □ 
With operators A and B, we can write the state equation (1.1) as follows: 

X = x + AX + Bu. 



(2.15) 



We now have the following result for the well-posedness of the state equaton. 



Proposition 2.6. Let (HI) hold. Then for any {x,u{- j) e R" x h{[0,T], state equation (1.1) admits a 
unique solution X{-) = X{-;x,u{-)) € X[0,T]. 

Proof. For any X{-) e X[0,T] and u{-) G U[0,T], by (2.9), we have 

e[ sup |(A;r)(s)p] <a(i)E[ sup \X{s)\^ 

with a{t) e (0, 1) when f > is small. Hence, by contraction mapping theorem, we obtain well-posedness 
of the state equation on [0,t]. Then by a usual continuation argument, we obtain the well-posedness of the 
state equation on [0, T]. □ 

From (2.9), we see that if for some £ > 0, 



A{-),A{-) G Li+"(0,T;R"^"), Ai(-), Ai(-) G ^2+^(0, T; R"^"), 
B{-),B{-) G L2(o,r;R"^™), Bi{-),Bi{-) G L°°(0, T; R"^™), 

7 



(2.16) 



then the result of Proposition 2.6 holds. It is ready to sec that (2.16) is stronger than (HI)" and weaker 
than (HI)'. In what follows, for convenient, we will assume (HI). However, we keep in mind that (HI) can 
actually be relaxed. 

Proposition 2.6 tells us that under, say, (HI), the operator I — A : X[0,T] — > A'[0,r] is invertible and 
the solution X to the state equation corresponding to (x, u{-)) € R" x W[0, T] is given by 



Note that 



Therefore, 



Now, let 



X = {I- A)-^x + (/ - A)-^Bu. 



{I - Ay^x + (7 - Ay^Bu = ItX = X{T) = x + AtX + Btu 



I + AT{I-Ay x+ At{I -A^B + Bt 



u. 



It{I - A)-^ =1 + At{I - A)-\ It{I - Ay^B = At{I - Aj'^B + Bt- 



(2.17) 



[QX{-)]{s) = Q{s)X{s), se[0,T], VX(.)eL|(0,T;]R"), 

[Q^{-)]{s) = QisMs), s e [0,T], Vifi-) G L2(0,T;R"), 

[TZu{-)]{s) = R{s)u{s), s G [0,T], Vm(-) G U[0,T], 

[TZ^i-Ms) = %)V(s), s e [0,T], VV(-) G l2(o,T;R™), 

g^ = G^, v^gXt, gx = Gx, yxewc-. 

Then the cost functional can be written as 

J{x; u{-)) = {QX, X)+{QEX, EX)+{-Ru, u)+{TiEu, Eu)+{gX{T), X{T))+{gE,X{T), EX(T)) 
= ( Q[{I - Ay^x + (7 - Ay^Bu], (7 - Ay^x + (7 - Ay^Bu ) 

+ ( QE[(7 - Ay^x + (7 - Ay'^Bu\,-E[{I - Ay^x + (7 - Ay^Bu] ) + {TZu,u) + { UEu, Eu ) 
+{0{[I+AT{I-Ay^]x+[AT{I-Ay^B+BT]u},[I+AT{I-Ay^]x+[AT{I-Ay^B+BT]u) 
+(gE{[7+^T(7-^)-']a;+[^T(7-^)-'B+BT]u},E{[7+^T(7-^)-']a;+[>^T(7-^)-'B+BT]u}) 
= ( 62U, u)+2{ Qix, u) + { Qqx, X ), 

where 

62 = 7e + E*7^E + B*{I - A*y^Q{I - Ay^B + B*{I - ^*)-iE*QE(7 - Ay^B 

-i{B*iI-A*y^A*T+B^]g[AT{I-Ay^B+BT] + [B*{I~A*y^A^+B^]]E*gw:[AT{I-Ay^B+BT] 

= n + E*:^E + B*{i - A*y\Q + e*qe)(7 - Ay^B 

+[B*{I - A*y^A*T + B*T]{g + E*^E)[^t(7 - Ay'B + Bt], 
Gi = B*{I - A*y^Q{I - Ay^ + B*{I - ^*)-iE*QE(7 - Ay^ 

+[S*(7 - A*y^Air + B*yg[i + At{I - A)y 

+[B*{I - A*y^A^ + S|,]E*aE[7 + At{I - Ay^] 
= B*{I - A*y\Q + E*QE)(7 - Ay^ + [B*{I - A*y^A*T + B^]{g + E*eE)[7 + Aril - Ay], 

60 = (7 - A*yQii - Ay + A*yM*m{i - Ay 

+[I + (7 - A*y^A^]g[I + At{I - A)y + [7 + (7 - A*y^A^]E*gE[I + At{I - A)-^] 
= (7 - A*y\Q + E*QE)(7 - Ay^ + [7 + (7 - A*y^A^]{g + E*gE)[7 + At{I - A)'^]. 



Consequently, for any u{-),v{-) € U[0,T], and x € R", 



(2.18) 



J{x;v{-)) = J{x;u{-) + [v{-)-u{-)]) 

= {e2[u + {v - u)],u + {v - u)) +2 {Qix,u + {v - u)) + {Qox,x) 
= { e2U, u)+2{ Six, u) + { Gox, x)+2{ Q2U + Qix, v-u) + { 62(1; -u),v-u) 
= J{x; u{-)) + 2 { @2U + Qix, V — u) + { Q2{v — u),v — u) . 

We now present the following result whose proof is standard, making use of the above (see [26] for details). 
Proposition 2.7. If u{-) ^ J{x; u{-)) admits a minimuni, then 

62 > 0. 



Conversely, if in addition to (2.19), one has 

eixee2(u[o,T]^, 

then u{-) i-> J{x;u{-)) admits a minimum u{-) G W[0,T]. Further, if 

62 > SI, 

for some S > 0, then for any given x G R", u{-) J{x; u{-)) admits a unique minimum. 
By the definition of 62, we see that (2.19) is implied by the following: 

TZ + E*nE>0, Q + E*QE>0, ^ + E*^E>0, 

and (2.21) is implied by 

n + m*nm>si, q + e*qe>o, g + m*gE>o, 



(2.19) 



(2.20) 



(2.21) 



(2.22) 



(2.23) 



for some 5 > 0. Now, we would like to present more direct conditions under which (2.19) and (2.21) hold, 
respectively. 

Proposition 2.8. Let (HI) and (H2)' hold. Then (2.19) holds. Further, if (H2)" holds for some 5 > 0, 
then (2.21) holds and Problem (MF) admits a unique solution. 

Proof. For any ^ e Xt, 
E[(G'^,C) + (GE[^],E[e])] =E 

and for any r] € Ut, 



>0, 



E 



{Q{s){^ - m),^- m ) + ([QW + Q(s)]EK],E[^] ) 



>o, 



E 



{R{s)v,v)+{R{s)nv],nv]) 



E 



( R{s){v - E[r?]), rj - E^ ) + {[R{s) + ii(s)]EM, E[r?] ) 



> 0. 



Thus, (2.19) holds. Next, if (H2)" holds, then 



E 



{R{s)ri,v)+{R{s)nv],nv]: 



E 



{R{s){v - mu- Hv] ) + {[R{s) + R{s)]nv],nv] : 



> ^E 



Hence, (2.21) holds. 



3 Optimality Conditions. 

In this section, we first derive a necessary condition for optimal pair of Problem (MF). 

Theorem 3.1. Let (HI) and (H2) hold. Let {X{-),u{-)) be an optimal pair of Problem (MF). Then 
the following mean-field backward stochastic differential equation (MF-BSDE, for short) admits a unique 
adapted solution {¥{■), Z{-)): 



dY{s) = - (A{sfY{s) + Ai {sfZ{s) + l(s)^E[r (s)] + li (s)^E[Z(s)] 

+Q{s)X{s) + Q(s)E[X(s)])ds + Z{s)dW{s), s e [0, T], 
Y{T) = GX{T) + GE[X(T)], 



(3.1) 



such that 



R{s)u{s) + B{sfY{s) + Bx{sfZ{s) + E(s)E[u(s)] + B{sf'E[Y{s)] + Bi(s)^E[Z(s)] = 0, 

se[0,T], a.s. 



(3.2) 



Proof. Let (X(-), m(-)) be an optimal pair of Problem (MF). For any u(-) € U[Q, T], let X(-) be the state 
process corresponding to the zero initial condition and the control u{-). Then we have 

= e| /" [ (Q(s)X(s),X(s) ) + (Q(s)E[X(.s)],E[X(.s)] ) + {R{s)u{s),u{s) ) 

+ { R{s)E[u{s)],-E[u{s)] )]ds + { GX{T), X{T) ) + { GE[X(T)], E[X(T)] ) } 

( Q{s)X{s) + Q{s)E[Xis)],X{s) ) + ( R{s)u{s) + R{s)E[u{s)], u{s) ) ] ds 

+ {GX{T) + G¥.[X{T)],X{T) ) }, 

On the other hand, by [9], we know that (3.1) admits a unique adapted solution {Y{-), Z[-)). Then 

E ( X{T), GX{T) + GE[X(T)] ) = E ( X{T), Y{T) ) 

= ^{J [{A{.s)X{s) + B{s)u{s) + A{s)]E[X{s)] + B{s)-E[u{.s)],Y{s)) 

- {Xis), AisfYis) + A.is fZis) + Ais fnYis)] + li(s)^E[Z(s)] + Qis)X{s) + g(s)E[X(s)] ) 
+ (^i(s)X(s) + Bi{s)u{s) + Ms)E[X{s)] + Bi{s)E[u{s)],Z{s) ) )ds} 

= e{ ( - ( X{s), Q{s)X{s) + g(s)E[X(s)] ) 

+ {u{s), Bi.sfYis) + Bii.sf Z{s) + Bisfm[Yi.s)] + Bi{.sfW][Z{.s)])y.sy 

Hence, 

Ej^(^{uis),B{sfY{s) + Bi{sfZ{s) + B{sfE[Y{s)]+Bi{sf^Z{s)]+R{^^^ = 0, 

which leads to 

R{s)u{s) + B{sfY{s) + Bi{sfZ{s) + R{s)E[u{s)] + B{sfE[Y{s)] + Bi{sfE[Z{s)] = 0. 



This completes the proof. 
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Prom the above, we end up with the following optimality system: (with s suppressed) 

' dX = (^AX + Bu + AE[X] + BE[u]J ds + (^A^X + B^u + AiE[X] + BiE[u]) dW{s), 
dY = - (a^Y + AlZ + QX + A^E[y] + E[Z] + OE[X]) ds + ZdW{s), ^3 3^ 

X(0) = X, Y{T) = GX{T) + GE[X(T)], 

L Ru + BM[u] + B^Y + B^Z + S^E[r] + E[Z] = 0. 

This is called a (coupled) forward-backward stochastic differential equations of mean-field type (MF-FBSDE, 
for short). Note that the coupling comes from the last relation (which is essentially the maximum condition 
in the usual Pontryagin type maximum principle). The 4-tuplc {X {■) , u{-) ,Y {■) , Z{-)) of F-adapted processes 
satisfying the above is called an adapted solution of (3.3). We now look at the sufficiency of the above result. 

Theorem 3.2. Let (HI), (H2), and (2.19) hold. Suppose {X{-),u{-),Y{-), Z{-)) is an adapted solution 
to the MF-FBSDE (3.3). Tiien {X{-),u{-)) is an optimal pair. 

Proof. Let {X{-),u{-),Y{-), Z{-)) be an adapted solution to the MP-PBSDE. Por any ■«(•) €U[0,T], let 

X,{.)=X{-;Q,u{-)-u{-)). 

Then 

X{s;x,u{-)) = X{s)+Xi{s), se[0,T]. 

Hence, (suppressing s) 

J{x;u{-)) - J(x;it(-)) 



ds 



{QX, Xi) + {QE-lX^-ElXi]) + {Ru,u - u) + {RE[u],'E[u - u]) 
+ { GX{T), Xi(T) ) + ( GE[X(r)], E[Xi(T)] ) } 

+e{ [ (QXi,Xi ) + (OE[Xi],E[Xi] ) + {R{u- u),u-u) + {RM[u - u],E[u - u\ ) 
+ { GX, (T) , Xi (T) ) + ( GE[Xi (T)] , E[X, (T)] ) } 



ds 



= 2E<^ 



{Xi,QX + QE[X] ) + {u-u,Ru + RE[u] )jds + {Xi(T), GX{T) + GE[X(r)] ) | 



+J(0; «(•)-«(•))• 



Note that 



Thus, 



E(Xi(r),GX(r) + GE[x(r)]) = E(Xi(T),y(r)) 

= e{ j (^{AXx+B{u-u)+AE.[Xx]+B^[u-u],Y) 

-{Xi, A^Y + AjZ + ^E[F] -I- If E[Z] +QX + QE[X] ) 
+ {AiXi+Bi{u-u)+Ai'E[Xi]+Bi'E[u-u],Z)^ds^ 

= ^{j (^-{Xi,QX + QE[X]} + {u-ii,B'^Y + B'[Z + B^E[Y]+B{E,[Z])ysy 



J{x;u{-)) - J{x;u{-)) 



= 2E 



[u-u,Ru + RE[u] + B^Y + BfZ + S'^E[F] + E[Z] ) ds + J(0; w(-) - u{-)) 



J(0; u{-) - u{-)) = { e2{u -u),u-u) >0. 
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Hence, {X{-),u{-)) is optimal. 



□ 



We have the following corollary. 

Corollary 3.3. Let (HI) and (H2)" hoJd. Then MF-FBSDE (3.3) admits a unique adapted solution 
{X{-),u{-),Y{-),Z{-)) of which {X{-),u{-)) is the unique optimal pair of Problem (MF). 

Proof. We know from Proposition 2.8 that under (HI) and (H2)", Problem (MF) admits a unique optimal 
pair {X{-),u{-)). Then by Theorem 3.1, for some {¥{■), Z{-)), the 4-tuplc {X{-),u{-),Y{-), Z{-)) is an adapted 
solution to MF-FBSDE (3.3). Next, if (3.3) has another adapted solution {X{-),u{-),Y{-), Z{-)). Then by 
Theorem 3.2, {X{-),u{-)) must be an optimal pair of Problem (MF). Hence, by the uniqueness of the optimal 
pair of Problem (MF), we must have 

x{-)^xi-), 2(0 = «(•)• 

Then by the uniqueness of MF-BSDE (3.1), one must have 

¥{.)=¥{.), Z{.) = Z{.), 
proving the corollary. □ 



4 Decoupling the MF-FBSDE and Riccati Equations 



From Corollary 3.3, we see that under (HI) and (H2)", to solve Problem (MF), we need only to solve MF- 
FBSDE (3.3). To solve (3.3), we use the idea of decoupling inspired by the Four-Step Scheme ([21, 22]). 
More precisely, we have the following result. For simplicity, we have suppressed the time variable s below. 

Theorem 4.1. Let (HI) and (H2)" hold. Then the unique adapted solution {X{-),u{-),Y {■), Z{-)) of 
MF-FBSDE (3.3) admits the following representation: 

u = -K-\B^P + BlPAi)(x - E[X]) - i^f 1 [(B + BfU + (Si + B^fP{A^ + Ai)] E[X], 
Y = P{X- E[X]j -h nE[X], 

PAi - PBiKo\b'^P + EfPAi) (jC - E[X]) 
P(Ai + li) - P{Bi + Bi)K^^ ((B + B)^n + (Bi + BifP{Ai + li))] E[X]. 



Z = 



+ 



where 



Ko = R + BfPBu Ki = R + R+{Bi+ BifP{Bi + Si) 
and P(-) and n(-) are solutions to the following Riccati equations, respectively: 

P' + PA + A^P -)- AjPAi -HQ - {PB+AIPBi)Kq^{B^P + BfPAi) = 0, 
P{T) = G, 



se[0,T], 



and 



u' + n 

+ 



{A + A)-{B + B)K^\B^ + Si)^P(Ai + li) 

{A + Af - (Ai + li)^P(Bi + Bi)K^\B + Bf\n- n{B + B)K^\B + Bfll 

P - PiBi + Bi)K^\Bi + Bi fp]{Ai + Ai) + Q + Q = 0, se[Q,T], 



+(Ai + Ai 

n(r) = G + G. 



(4.1) 



(4.2) 



(4.3) 



(4.4) 
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Finally, X{-) solves the following closed-loop system 



dX = 



{ 



{A + A)-{B + B)K^^ ({B + Bfn + (Bi + BifP{Ai + li))]E[X]}dt 
+ { [^1 - BiKo\B^P + BlPAi)j (X - E[X]) 



(4.5) 



+ 



(Ai + li) - (Bi + Bi)i^r' ((S + BfU + (Bi + BifP{Ai + Ai))] E[X] 



[ X(0) = X. 



Proof. First of all, under (HI) and (H2)", Riccati equation (4.3) admits a unique solution P(-) which 
is positive definite matrix valued ([34]). With such a function P{-), Ki defined in (4.2) is positive definite. 
Next, we note that 

P - P{B, + Bi)K^\B, + B,fP 

= P-P{Bi+Bi) \R+R+{Bi+BifP{Bi + Bi)\ ^{Bi+BifP = P-PB{R+B'^PB)-^B'^P 



= P2 



7-r(7+r'^'r)-^r'^ 



pi 



I-P^BR-i {i+R~^B'^P^P^BR~^^ ^R-iB'^pi 
= P^{I+TT^-'^pi=pi{I+piBR-^B'^pi)-'^pi=pi[I+pi{Bi+Bi){R+R)-\Bi+B^^ 

In the above, we have denoted 

B = Bi+Bi, R = R + R, r = piBRi, 

and used the fact 

7 _ r(/ + r^r)-ir^ = (/ + r^r)-\ 

Hence, Riccati equation (4.4) is equivalent to the following: 



H' + H 



{A + A)-{B + B)K^\Bi + BifP{Ai + Ai) 



+ 



{A+Af-{A^+AifP{Bi+Bi)K^\B+Bf]u-UiB + B)K^\B+BfU 



(4.6) 



Since 



+ {Ai+AifP^\l+P^{Bi+Bi){R+R)-\Bi+BifP^^ P^Ai+Ai)+Q+Q = 0, 
n(T) = G + G. 

G + G>0, 

iAi+Aifpi^I+pi{B^+Bi)iR+R)-\Bi+BifpiY^ pi{Ai+Ai)+Q +Q >0, 

according to [34], Riccati equation (4.4) admits a unique solution H(-) which is also positive definite matrix 
valued. 



Now, suppose {X{-),u{-),Y{-), Z{-)) is the adapted solution to (3.3). Assume that 

Y{s) = P{s)X{s) + P(s)E[X(s)], s G [0, T], 

for some deterministic and differentiable functions P(-) and P(-) such that 

P(T) = G, P(T) = G. 
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(4.7) 



(4.8) 



For the time being, we do not assume that P(-) is the solution to (4.3). Then (suppressing s) 

( - A^Y - AlZ -QX- I^E[r] - if E[Z] - gE[X])ds + ZdW{s) 
= dY = d(px + PE[X]^ 



P'X + P(^AX + Bu + A¥][X] + BTE][u]j + P'E[X] +p(^{A + A)E[X] + {B + B)E[u]) 
+p(^AiX + Biu + iiE[X] + BiE,[u]jdW{s) 



ds 



ds 



= (P' + PA)X + PBu + (P' + P{A + A) + PA^ E[X] + (P{B + B) + PBj E[u] 
+p(^AiX + Biu + liE[X] + BiE[u])dVF(s) 

Comparing the diffusion terms, we should have 

Z = p(AiX + B^u + liE[X] + BiE[«]). 

This yields from (3.2) that 

Q = Ru + ]m[u\ + B^Y + B^E[r] + B'[Z + E[Z] 
= Ru + RE[v] + B'^{PX + PE[X]) + B^ {P + P)E[X] 

+Sfp(^iX + Biu + liE[X] +PiE[u]) +i3fp((^i +li)E[X] + (Pi +Pi)E[u]) 
= (P + BfPBi)u + {R + BfPBi + BfPBi + Pf PPi)E[u] + (P'^P + BfPAi)! 

+ (b'^P + B^{P + P) + BfPAi + B{P{Ai + 1i))e[X]. 

Taking E, we obtain 

P+P+(Pi + Pi)^P(Pi + Pi)]E[u]+[(P + P)^(P+P) + (Pi+Pi)^P(^i+ii)]E[X] 
= /<:iE[u]+[(P+P)^(P+P) + (Pi+Pi)^P(Ai+ii)]E[X]. 

Assuming 

i^i = i^i(P) = P + P + (Pi + Pi)^P(Pi + Pi) 

to be invertible, one gets 

TE[u] 

Then we have 

= (P + BfPBi)u + (P + Pf PPi + Pf PPi + Pf PPi)e[u] + (P^P + B'[PAi)X 

+ (b'^P + B'^{P + P)+B^PAi + B'[P{Ai+Ai))e[X] 



-k: 



(P + P)^(P + P) + (Pi + Pi)^P(Ai + Ai) E[X]. 



(4.9) 



(4.10) 



= K^u + (P^P + BlPAi)X + +[B'^P + B^{P + P) + bIpAi + B{P{Ai + Ai)jE[X] 
- (R+BlPBi+BfPBi+BlPBi^ K^^ [(P+P)^(P+P) + (Pi +Pi)^P(Ai + Ai)] E[X]. 

Consequently, by assuming 

Kq = Ko{P)=R + BJPB^ 
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to be invertible, we obtain 



-Kq^{B^P + Bj'PAi)X - K-^ (^B^P + B^{P + P) + B^PAi + P(Ai + 1i))e[X] 

^Ko^(R+BlPBi + B^PBi + B^PB^K^^\^{B + Bf{P+P) + {Bi + BifPiAi+A^^^ 



= -K-\B^P + BlPAi)X - K-^ [b^P + B'^iP + P) + B^PAi + + 1i))e[X] 

+K^^ (Ki - Kq^K^^ [{B + Bf{P + P) + (Bi + BifP{Ai + Ai)] E[X] 
= -Kq^B'^P + PAi)X - Kq^ {b'^P + B'^{P + P) + -Bf p1i + Pf P(Ai + Ai))e[X] 



(4.11) 



-k: 



(P + Bf{P + P) + (Pi + Pi)^P(Ai + Ii) 
(P + P)^(P + P) + (Pi + Pi)^P(Ai + li) 



E[X] 
E[X] 



-Kq^{B^P + Bj'PAi)X + K^^ B'^P + B'fPAi E[X] 



-iff 



(P + P)^(P + P) + (Pi + Pi)^P(Ai + Ai) E[X]. 



Here, we note that 

Ki-Ko = R + R + {Bi+ BifP{Bi + Pi) - P - Pf PPi 
= R + BfPBi + BfPBi + BfPBi. 

Hence, comparing the drift terms in (4.9), we have 

= (P' + PA)X + PBu + (P' + P{A + A) + PAj E[X] + (p(P + P) + PP) E[u] 
+A'^(^PX + PE[X]) + Ajp(^AiX + Biu + AiE[X] + PiE[u]) 
+^(P + P)E[X] + ^P [(Ai + li)E[X] + (Pi + Pi)E[u]] +QX + QE[X] 

= P' + PA + A^P + AlPAi +q\x + {PB + AlPBi)u 

+ [p'+P(A+l) + (A+l)^P+Pl+l^P+(Ai + li)^P(Ai+Ai)-AfPAi + Q]E[X] 
P(P + P) + PP + PPi + P(Pi + Pi)l E[u] 



+ 



P' + PA + A^P + A^PAi + Q 



X 



+{PB + PPi)| - Ko\B^P + BfPAi)X + Kq^ B^P + BfPAi E[X] 
-K^^ [(P + P)^(P + P) + (Pi + Pi)^P(^i + Ai)] E[X]} 



+ 



P'+PiA+A) + {A+AfP+PA+A:^P+{Ai+AifP{Ai+Ai)-A^^PAi+Q E[X] 
P(P + B) + PB + AjPBi + If P(Pi + Pi) 



•{ - iff 1 [(P + P)^(P + P) + (Pi + Pi)^P(Ai + Ai)] E[X]} 



= [P' + PA + A^P + Af PAi +Q- {PB + A^PBi)Kq^{B'^P + BfPAi) 

+{P' + P{A + A) + {A + AfP + PA + A^P + (Ai + MfP{A^ A^PA^ + Q 

+{PB + A'[PBi)Ko\B^P + BfPAi) 

- (P+P)(P+P) + (Ai+li)^P(Pi+Pi)]iffi[(P+P)^(P+P) + (Pi+Pi)^P(Ai+Ai)]}E[X]. 

Therefore, by choosing P(-) to be the solution to Riccati equation (4.3), we have that Kq and Ki are positive 
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definite, and the above leads to 



0. 



|p' + p(A + A) + {A + AfP + PA + I^P + {Ai + AifP{Ai + Ai) - AjPA^ + Q 

+ {PB + A{PBi)K^\B'^P + B^PAi) 

- (P+P)(B + B) + (Ai + li)^P(Bi + Bi)]i^i-i[(B + B)^(P + P) + (5i + Bi)^P(yli+li)]}E[X] 
Now, if P(-) satisfies the following: 

Q^P' + P{A + A) + {A + AfP + PA + A^P + {Ai + AifP{Ai + Ai) - AjPAi + Q 
+ {PB + AJPBi)Ko\b'^P + BfPAi) 

- (P + P){B + B) + {Ai + AifP{Bi + Pi)] ^ [(P + P)^(P + P) + (Pi + BifP{Ai + Ai) 
= P' + P [a + 1 - (P + B)K^^ ((P + BfP + (Pi + Pi)^P(Ai + Ai)) 

+ [{A + Af - (P(P + P) + (Ai + AifP{Bi + Bi)^K^\B + P)^]p - P(P + B)K^\B + P)^P 
+PA + A^P + {At + MfP{At + M)- A^PAi + Q + {PB + AlPBt)K^\B'^ P + Pf PAi) 
P(P + P) + (Ai + Ii)^P(Pi + Pi)] isTfi [(P + P)^P + (Pi + Pi)^P(Ai + ii)] , 

then (F(-), Z{-)) defined by (4.7) and (4.10) with u{-) given by (4.11) satisfies the MF-BSDE in (3.3). Hence, 
we introduce the following Riccati equation for P{-): 



P' + P 



A + A-{B + B)K^^ ( (P + P)^P + (Pi + Pi)^P(Ai + Ai) 



+ 



{A + Af - (P{B + B) + (Ai + Ai)^P(Pi + Bt))K^\B + Bf 



-P{B + B)K^\B + BfP + PA + A^P + {A^ + AifP{Ai + Ai) - AjPAi + Q 
+{PB + AlPBi)Ko\B^P + BlPAi) 



(4.12) 



(P + P)^P + (Pi + Pi)^P(Ai + Ai) 



= 0, 



- P(P + P) + (Ai + Ai)^p(Pi + Pi) 

[ P{T) = G. 

The solvability of this Riccati equation is not obvious since G is just assumed to be symmetric, and 

Q = PA + A^P + {Ai + MfP{Ai + M) - AjPAi + Q + (PP + AJpBi)K^\B^P + BfPAi) 
- P{B + B) + (Ai + Ai)^P(Pi + Pi)] i^fi [(P + P)^P + (Pi + Pi)^P(Ai + M) 

is also just symmetric. To look at the solvability of such a Riccati equation, we let 

n = P + P. 

Then 

n(r) = G + G > 0, 
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and consider the following 



= P' + PA + A^P + A^PAi + Q- {PB + A'[PBi)Kq\b'^P + BfPAi) 

+P' + p(^A + A) + {A + AfP + PA + A^P + {Ai + AifP{Ai + Ai) - AjPAi + Q 
+iPB + AlPBi)K^\B'^P + BfPAi) 

- {P+P)iB+B) + iAi+AifP{Bi+Bi)\K{^[{B+Bf{P+P) + iBi+BifP{Ai+Ai) 
= n'+P{A+A) + iA+AfP+Q+Q+{Il-P){A+A) + {A+Af{Il-P) + iA,+AifP{Ai+Ai) 



- n{B + B) + {Ai + AifP{Bi + Bi) 

n' + n 



{B + Bfn + (Si + BifP{A^ + Ai] 



{A + A)-{B + B)K^\Bi + BifP{Ai + Ai) 
{A + Af - {Ai + Aif P{Bi + Bi)K-\B + Bf]u - U{B + B)K-\B + BfU 
+{Ai + Aif [P - P{Bi + Bi)K^\Bi + Bifp] (Ai + Ai) + Q + Q. 

Thus, n(-) is the solution to Riccati equation (4.4). Consequently, Riccati equation (4.12) admits a unique 
solution P(-) = n(-) - P(-). Then we obtain (from (4.7) and (4.11)) 



Y = p(x - E[X]) + nE[X], 



and 



u = -Kq\b'^P + BfPAi)(x - E[X]) - K^^ [(B + Bfll + (Si + Bi)^P(Ai + li)]lE[X]. 
Also, from (4.10), it follows that 

Z = p(^AiX + Biu + AiE[X] + PiE[u]) 

= PAiX-PSi{i^o-i(B^P+BfPAi)(x-E[X])+i^fi[(P+P)^n+(Si+Si)^P(Ai+ii)]E[X]} 

+PAiE[x]-pPiii:fi[(p+p)^n+(Pi+Pi)^p(Ai+ii)lE[x] 

= PAi-PBiKq\B^P + B'[PAi) (^X-TEIX]^ 

+ [p(Ai + Ai) - p(Pi + Bi)i^f 1 ((P + P)^n + (Pi + Pi)^p(Ai + li))] E[x]. 

Hence, (4.1) follows. Plugging the above representations into the state equation, we obtain 

dX = ^^AX-B Xo-\p'^P+PfPAi)(x-E[X])-Xfi((P + P)^n+(Pi+Pi)^P(Ai+li))E[X] 
+1e[X] - Pi^f 1 (P + BfU + (Pi + Pi)'^P(Ai + li) E[X]}dt 

+ {AiX-Pi [/s:o"'(S^^'+SfPAi)(x-E[X])-i^fi((P+P)^n+(Pi + Pi)^P(Ai+ii))E[X] 



+AiE[X] - BiK^ 



{ 



+ 



■{ 



+ 



(p + Bfn + (Pi + Pi)^P(Ai + li)]E[x]}dW(t) 

A-BKq'^{B'^P + BIPAi) (^X-E[X]) 

(A + 1) - (P + B)K^^ ((P + P)^n + (Pi + Pi)'^'P(Ai + ]E[X] 

Ai- BiKf^^iB'^P + B^PAi) (^X-]E[X]^ 
(Ai + li) - (Pi + Bi)K-^ ((P + P)^n + (Pi + Pi)^P(Ai + li))]E[X]}dM/(t). 



dt 
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This gives the elosed-loop system (5.15). From the above derivation, we see that if X{-) is a solution to 
(5.15), and {u(-),Y{-), Z{-)) are given by (4.1), then {X{-),u(-),Y{-), Z{-)) is an adapted solution to MF- 
FBSDE (3.3). By Corollary 3.3, we know that such a constructed 4-tuple {X{-),u{-),Y{-), Z{-)) is the unique 
solution to (3.3). □ 

The following is a direct verification of optimality of state feedback control. 

Theorem 4.2. Let (HI) and (H2)" hold. Let P{-) and n(-) be the solutions to the Riccati equations 
(4.3) and (4.4), respectively. Then the state feedback control u{-) given in (4.1) is the optimal control of 
Problem (MF). Moreover, the optimal value of the cost is given by 

inf J(x;u(-)) = (ll(0)x,x}, Vx G R". (4.13) 
ui-)euio,T] 

Proof. Let P(-) and n(-) be the solutions to the Riccati equations (4.3) and (4.4), respectively and denote 
Kq and Ki as in (4.2), which are positive definite. Let P(-) = n(-) - P(-). Then P(-) solves (4.12). Now, 
we observe 

J{x;u{-)) - {U{0)x,x) = J{x;u{-)) - {[P{0) + P{0)]x,x) 

= Ej {{QX,X) + {QE[X],E[X]) + {Ru,u) + {m[u],-E[u]) 

+ { P'X, X)+2{ P{AX + Bu + 1e[X] + B'E[u]),X ) 

+ ( PiAiX + Biu + AiE,[X] + BiE[u]), A^X + B^u + A{E{X\ + BiE[u] ) 

+ ( P'E[X], E[X] ) +2 (PE[X], {A + A)E[X] + (B + B)E[«] ) }rfs 



E 



\ I {{iP'+PA+A'^P+A'[PAi+Q)X, X )+2 ( w, (P^P+PfPAi)X )+{{R+BfPBi)u, u ) 

+ ( QE[X],'E[X] ) + ( m[u],'E[u] ) +2 ( P{AE[X] + SE[u]), E[X] ) 

+2 (P(AiE[X] + BiE[u]),1iE[X]+PiE[m]) + (P(1iE[X]+PiE[u]),AiE[X]+BiE[m]) 

+ ( P'E[X], E[X] ) +2 ( PE[X], {A + A)E[X] + {B + 5)E[u] ) }ds 

E j [{iP'+PA+A'^P+AjPAi+Q)X,X)+2{u,{B^P+BfPAi)X)+{Kou,u) 

+ {[P'+PA+A'^P+A^PAi+A{PAi+A^PAi+P{A+A) + {A+AfP+Q]E[^^^ ) 

+ {{R + BfPBi + BfPBi + PBi)E[u], EM ) 

+2 (EM, [B^P + BlPAi + (Bi + BifPAi + (B + BfP]E[X] ) }rfs 

E j {{iP' + PA+A'^P+AjPAi+Q)X,X)+2{u-TE[u],{B'^P+BlPAi){X--E[X])) 

+ (Xo(u"EM),w-EM) 
+ {(P' + PA + A^P + S[PAi + A{PAi + A^PAi 

+P{A + A) + {A + AfP + g)E[X], E[X] ) + ( i^iEM, EM ) 

+2 (EM, {B + Bf{P + P) + (Si + BifPAi + {Bi + BifPAi^E[X] ) }ds 
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= E ^ \^{[P'+PA+A'^P+AjPAi+Q-{PB+AiPBi)Ko\B'^P+BlPAi)]X, X ) 
+ kI \u - E[u] + Ko\b'^P + BIPAi){X - E[X])1 ^ 

+ ( [p' + PA + A^P + A^PAi + AjPAi + A^PAi + P{A + A) + {A + AfP + Q 

+iPB + AiPBi)Ko\B^P + BfPAi) 

- ((P + P)(P + B)+ P(Bi + Bi) + ^P(Bi + Pi))^s:r' 

E[X],E[X]} 



K 



= E 



((P + Bf{P + P) + (Pi + BifPAi + (Pi + Pi)^PAi 

]E[u] + K^^ (^{B+Bf (P+P) + (Pi +Pi)^PAi + (Pi + Pi)^p1i) E[X]] | ' }ds 

\u - E[u] + Ko\b'^P + B'[PAi){X - E[X])1 ^ 

A7 [e[u] + i4:r^((P + BfU + (Pi + BifPAi + (Pi + Pi)^PAi)e[X]] ^jds > 0. 



{ 



Then our claim follows. 

We sec that Riccati equation (4.4) can be written as follows: 

^n' + n(A + A) + {A + A)^n + {a^ + 1i)^p(Ai + + q + q 

- n(P + P) + (Ai + li)^P(Pi + Pi)] K^^ [(P + Bfn + (Pi + Pi)^P(>li + Ai) 

sG [0,T], 

U{T)=G + G. 



0, 



When 
we have 



A = Ai=0, P = Pi = 0, 



Q = 0, 



R = 0, G = 0, 



(4.14) 



/^To = i^i = P + PfPPi, 
and the Riccati equation for n(-) can be written as 

n' + n^l + A^U - (HP + A'[PBi)Ko\B^U + BfPAi) + A{PAi + Q = 0, 
n(T) = G. 
Then 

= (n - P)' + (n - P)A + A'^{Il - P) - (HP + A'[PBi)Kq\B^U + B'[PAi) 
+{PB + AIPBi)Kq\B^P + B^PAi) 

= (n - py + (n - p)a + A^(n - p) - (n - p)Piv'o"^(p'^n + BfPAi) 

-{PB + AlPB-i)K^'^B'^{n - P). 
Therefore, by uniqueness, we have 

n = P. 

Consequently, the fccdback control can be written as 

u = -Ko\B^P + BlPAi)(^X - E[X]) - i^f ^ [(P + BfiL + (Pi + Pi)^P(Ai + li)] E[X], 

= -Kq\b'^P + B'[PAi)(x -'E[X]^ -KQ'^(B^P + BfPAi^'E[X] 

= -Ko\b'^P + B'[PAi)X. 
This recovers the result for classical LQ problem ([34]). 
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5 A Modification of Standard LQ Problems. 



In this section, we are going to look at a special case which was mentioned in the introduction. For conve- 
nience, let us rewrite the state equation here: 



dX{s)= A{s)X{s) + B{s)u{s) ds+ Ai{s)X{s) + Bi{s)u{s) dW{s), 
X{0) = X, 
with the cost functional: 



Classical LQ problem can be stated as follows. 

Problem (LQ). For any given x € R", find a u(-) G U[0,T] such that 

Jo{x;u{-))= inf Jo{x;u{-)). 

M(-)ew[o,T] 



+ PoA + A'^Po + AjPoAi + Qo 
-iPaB + AlPoBi){Ro + B^PoBi)-\B^Po + B^PoAi) = 0, s e [0,T], 
[ Po{T) = Go. 

and X{-) is the solution to the following closed-loop system: 

dXo{s) = [a- B{Ro + B'[PqBi)-^{B'-^'Pq + B'[PQA)\x{s)ds 

+ [Ai-Bi{Ro + B^PoBi){B^Po + BlPoAi)^Xis)dWis), s G [0,r], 
^Xo(O) = x. 

We now introduce the following modified cost functional: 
Jo{x;u{-)) = lE 



+E 



E 



( ( Qo{s)X{s), X{s) ) + ( Ro{s)u{s), u{s) ))ds + { GoX{T),X{T) ) 

(^g(s)var [X{s)] + p(s)var [u{s)]jds + gvai [X{T) 

[< {Qo{s) + q{s)l')x{s),X{s) ) -9(s)|e[X(s)] p + ( (i?o(s) + p{s)l)u{s), u{s) ) 
p(s)|E[w(s)]|']ds + ( (Go +ff/)x(T),X(r) ) -5|e[X(T)]|'}, 



(5.1) 



Mx; u{-)) = E [ ( ( Qo{s)X{s),X{s) ) + ( R<i{s)u{s), u{s) ))ds+{ GqX{T),X{T) ) ] . (5.2) 



(5.3) 



The following result is standard (see [34]). 
Theorem 5.1. Let (HI) hold and 

Qo(s)>0, Rq{8)>5I, sG[0,r]; Go > 0. (5.4) 

Then Problem (LQ) admits a unique optimal pair {Xq{-),uo{-)). Moreover, the following holds: 

uois) = -[Ro{s) + Bi{sfPois)B{s)Y\B{sfPo{s) + Bi{sfPo{s)Ai{s)]Xois), s€[0,T], (5.5) 
where Po{-) is the solution to the following Riccati equation: 



(5.6) 



(5.7) 



(5.8) 
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with q{-),p{-) e L°°{0,T), g G [0, oo) such that 

g(s),p(s)>0, se[0,T]. 

Also, of course, we assume that 

/ + p(s)]ds + 5 > 0. 

Jo 

We want to compare the above Problem (LQ) with the following problem: 
Problem (LQ)'. For any given x G R", find a u{-) G U\Q,T] such that 

Jo(x;u(.))=^^^inf^_^jJo(x;u(.)). 



(5.9) 



(5.10) 



(5.11) 



We refer to the above Problem (LQ)' as a modified LQ problem. This is a special case of Problem (MF) with 

1 = Ai = 0, B = Bi=0, 

Q^Qo + ql, Q = -qI, R = Ro + pI, R = -pI, G = Go+gI, G = -gI. 
Then the Riccati equations are 

P' + PA + A^P + A^PA^ + Qo+qI 

-{PB + A'[PBi){Ro+pI + B^PBi)-\B^P + B^PAi) = 0, s G [0,T], (5.12) 
[P{T) = Go + gI, 

and 

U'+IlA+A^n+AjPAi + Qo-{UB+A^PBi){Ro+BfPBi)-\B^n+B]^PAi) = 0, s G [0,T], 
n(T) = Go. 

The optimal control is given by 

u = -(i?o + pl + BjPBiy^B^P + B'[PAi)(^X - E[X]j 
-{Ro + B{PBi)-\B^IL + BlPAi)E[X], 

and the closed-loop system reads 

dX=| A- B{Ro + pi + B'[PBi)-\b'^P + B'[PAi) (^X -lE[X]j 
+ A^ B{Ro + BfPBi)-^(^B^Il + BfPAi^ E[X]|ds 
+ { [^1 - BiKo\B^P + BlPAi)\ (X - E[X]) 
+ Ai-BiK^^(^B'^U + B'[PAi^ E[X]}dVF(s), 
X{0) = X. 

By the optimality of zto(-) and u(-), we have that 

{Po{0)x,x) = Mx;uo{-)) < Mx;u{-)), (5.16) 



(5.13) 



(5.14) 



(5.15) 



21 



and 



{U{0)x,x) = Jo(x;u(-)) + E (q{s)Yai [X {s)] + p{s)vaT [u{s)]^ ds + gvai [X {T)] 
= Jo{x;u{-)) < Jo{x;uo{-)) 

= Jo{x; uo{-)) +^ (g(s)var [Xo{s)] + p(s)var [uo{s)]jds + gvai [Xo{T)] 
< Jo{x; «(•)) +^ (q{s)vai [Xo{s)] + p(s)var [Mo(s)])ds + gvar [Xo{T)] 
This implies 



(5.17) 



E 



< E 



(^g(s)var [X{s)] + p{s)var [u{s)fjds + gvar [X{T)] 
/ (q{s)vaT [Xo{s)] + p(s)var [uq 



(5.18) 



Hence, 



Jo{x;u{-)) - Jo{x;uo{-)) 



is the price for the decrease 



E 



-E 



(§(s)var [Xo{s)] + p(s)var [uo{s)]^ds + gvav [Xo{T)] 

T 

(g(s)var [X{s)] + p(s)var [m(s)]) ds + gvar [X{T)] 



of the (weighted) variances of the optimal state-control pair {Xq{-),uo{-)). Moreover, (5.17) further implies 
that 

Jo{x;u{-)) - Jo(x;uo(-)) 



^ (g(s)var [Xo(s)] + p(s)var [uo{s)\^ds + fifvar [Xo(r)] 
{q{s)Yav [X{s)\ + /9(s)var [u(s)])ds + fiivar [X(T)] 



< E 



The above roughly means that the amount increased in the cost is "covered" by the amount decreased in 
the weighted variance of the optimal state-control pair. 

We now look at a simple case to illustrate the above. Let us look at a one-dimensional controlled linear 
SDE: 

dX{s) = bu{s)ds + X{s)dW{s), 
X{0) = X, 

with cost functionals: 



(5.19) 



Jo{x;u{-)) = B / \u{s)fds + go\X{T)\' 



and 



Jo{x;u{-))^]e{J^ \u{s)\^ds + go\X{T)\^+gyar[X{T)]} 

= e{ £ \uis)fds + {go + 9)\X{T)f - 5(e[X(T)])'}, 



(5.20) 



(5.21) 
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where go > and g > 0. As above, we refer to the optimal control problem associated with (5.19) and (5.20) 
as the standard LQ problem, and to that associated with (5.19) and (5.17) as the modified LQ problem. The 
Riccati equation for the standard LQ problem is 



p'ois) + Po{s) - b^poisf = 0, se[0,T], 
Po{T)=go. 



T- „ 

e 50 



and the closed-loop system is 



dXois) = -b''po{s)Xo{s)ds + XQ{s)dW{s), s £ [0,T], 
Xo(0) = X. 



Thus, 



Consequently, 



and 



Hence, 



p'{s)+p(.s)-b^pisf = 0, SG[0,T], 

p{T) =go + g, 
and 

^'(s) -&V(s)+p(s) = 0, se[o,r], 

tt{T) = go- 
Clearly, 



(5.22) 



A straightforward calculation shows that 

Ms)= ^^r_:_,-g^^, >0, se[0,T]. (5.23) 



The optimal control is 

uo{s) = -bpois)Xois), se[0,T], (5.24) 



(5.25) 



E[Xo(T)] = xe-''Tf°(^)'^^-^^+^^ = xe-'^rf^^^)"^^, (5.27) 

E[Xo(T)2] = x^e-^'^ />o(r)dr-T+2T ^ ^2^-26^ />o(r)dr+T_ ^^ ^S) 



var [Xo(T)] = E[Xo{Tf] - (e[Xo(T)])' = x^e"'^' ^^"^^^''"(e^ - 1). (5.29) 
Also, the optimal expected cost is 

Mx;uoi-))=PoiO)x' = , ^ ^"^g x\ (5.30) 

(e^ - 1)0^50 + 1 

Next, for the modified LQ peoblem, the Riccati equations are: 



(5.31) 



(5.32) 



We now show that 

Po{s) <nis) <p{s), se[0,T]. (5.34) 
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In fact, 



Thus, 



Next, 



— [^(s) -po{s)] - b^[TT{s)+po{s)Ms)-pois)] +pis) -pois) =0, s& [0,T], 
Tr{T)-po{T)=0. 

^{s) -Po{s) = r e-Il'^^^^^^+'^^^^^'^m-Poimt > 0, s€ [0,T). 

J s 

A. _ ^(s)] _ + ^(s)] _ 7,(5)] =0, s G [0, T] , 

p{T) - n{T) = 5. 



Hence, 



p{s) - 7r(s) = e-r'''Wr)+-(r)]dr^ ^ ^ ^ 

This proves (5.34). Note that the optimal control of modified LQ problem is given by 
u{s) = -bp{s) (X{s) - E[X(s)]) - 67r(s)E[X(s)], s e [0, T], 
and the closed-loop system is 

dX{s) = - b'^p{s)(^X{s) - E[X(s)]) + 6V(s)E[X(s)] ds + X{s)dW{s), s € [0,T], 
X{Q) = X. 



Thus, 



d_ 

ds ' 



which leads to 



■[nx{s)]) = -h\{s)nx{s)i 

E[X(s)] = e-*"' /o' "(^)'^^a;, s G [0, T]. 
On the other hand, by Ito's formula, 

d[X^] = { - 2 h^px{x - E[X]) + ft^TrXEfX] + X'^^ds + [• ■ -JdW. 



Then 



Hence, 



d(E[X2]) = { -262p[E[X2] - (e[X])'] ~2fe27r(E[X])' + E[X2]|ds 
= {(1 -262p)E[X2] +262(p-7r)(E[X])^}(is 
= {(1 - 262p)E[X2] + 262(p - 7r)e-''' /o' "(^'''Vlrfs. 

E[X2(s)] = e*-2fV;fM'iv[l+ /'e-*+'./'o^'(^)'^^26Xi) -7r(i)]e-"''-/'o '^(^^''^ 

L io 

L Jo 

L io 

-26= Jo '^(-r)<ir _^ r gS-t-26= p(r)dr-26= ,r(r)dr^ 
JO 



(5.35) 



(5.36) 
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It follows that 



var[X(s)] = 



Consequently, noting P(i{s) < p{s), we have 

var [X{T)] = \ / V"*"'"' p(r)<ir-26= /; po(r)dx^^- 

L Jo 

On the other hand, we claim that 



dt 



/•J 

T)dT 

Jo 



(5.37) 



e'^-'dt 



=var[Xo(r)]. 



Po{s) > 



9o 



90+9 



p{s), s€[0,T). 



(5.38) 



In fact, by letting p = -^^^P, we have 



ff{s)+pis)-bMsf 
P{T) = 90. 



909 



{90+9? 



p{sf = 0, 



Then 



[Po(s) -P'(s)] + \po{s)-p{s)\ - &'[Po(s) +p{s)\\Po{s)-p{s)\ + 



909 



{90 + 9) 



;p{sf = 0, 



^ Po(T)-p(r) = 0. 
This leads to (5.38). Consequently, 

Jo(a;; «(•))+ var [X(T)] = Mx;u{-)) = n{0)x^ <p{0)x^ < i^po{0)x^ = i°±^J{x;uoi-)). 

9o 9o 



Therefore, 



< Mx; ui-)) - Mx; uo{-)) + gvar [X{T)] < ^J{x- wo(-))- 

.90 



The above gives an upper bound for the cost increase in order to have a smaller var [X{T)]. Taking into 
account of (5.37), we see that it is a very good trade-off to consider the modified LQ problem if one wishes 
to have a smaller var[X(T)]. It is possible to more carefully calculate the price difference Jq{x;u{-)) — 
Jq{x;uq{-)). We omit the details here. 

Also, it is possible to calculate the situation of including var [u(s)] and/or var [X(s)] in the integrand of 
the modified cost functional. The details are omitted here as well. 

To conclude this paper, let us make some remarks. First of all, we have presented some results on the 

LQ probk^m for MF-SDEs with deterministic coefficients. Optimal control is represented by a state feedback 
form involving both X{-) and ]E[X(-)], via the solutions of two Riccati equations. Apparently, there arc 
many problems left unsolved (and some of them might be challenging). To mention a few, one may consider 
the case of infinite- horizon problem, following the idea of [31], and more interestingly, the case of random 
coefficients (for which one might have to introduce some other techniques since the approach used in this 
paper will not work). We will continue our study and report new results in our future publications. 
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